Image sequence processing addresses problems such as interframe motion estimation, enhancement, restoration and data compression of time-varying image sequences. In the last decade, there has been a continuous research effort in motion estimation, enhancement and data compression of image sequences.
An important application of digital image sequence restoration is the deblurring of a sequence of images that suffer from motion and/or focus blur as well as noise contamination. Motion blur is common in images recorded by high-speed video cameras used in industrial and scientific applications that require monitoring of high-speed events, surveillance cameras tracking moving objects, and cameras mounted on a moving vehicle such as an aircraft. Blur due to defocus, on the other hand, may also occur in all these cases. It is of common occurrence, however, in autofocus video cameras where images may often be blurred due to defocus caused by the inertia of the focusing system.
Restoration of a blurred and noisy image sequence potentially increases the amount of information that a human observer can obtain from an image sequence. In cases where the blur and noise degradation is not perceivable due to motion, restoration may increase the efficiency of any subsequent processing that is applied to the image sequence, such as freeze-frame display, analysis, and data compression.
One well-known technique of restoration that has been performed on a single image is known as Wiener restoration. A trivial extension of single-image (or single-frame) Wiener restoration to an image sequence is obtained by treating each frame as an independent single image and separately restoring them. This single-frame approach, however, does not make use of the information contained in the other frames in restoring a particular frame. In the context of Wiener restoration, this amounts to disregarding the existing statistical correlation between the image frames. An alternate approach, that does take into account the interframe correlation, is the multiframe approach where the problem of simultaneous restoration of multiple frames is addressed.
A multispectral Wiener restoration filter for restoring images with multiple spectral bands is known from "Digital Restoration of Multichannel Images", IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-37, pp. 415-421, (March 1989).
This Wiener filter structure has also been applied to the restoration of a sequence of misregistered radar imagery by Srinavas and Srinath in "A Stochastic Model-Based Approach for Simultaneous Restoration of Multiple Misregistered Images", SPIE, vol. 1360, pp. 1416-1427, (1990). In that proposed filter structure, the multiple images are treated as image data at different spectral bands.
A problem with the prior art is the inefficient computation of the cross-correlated multiframe Wiener estimate. The Wiener estimate normally requires the inversion of an NM.sup.2 .times.NM.sup.2 matrix, where M.sup.2 is the number of total pixels in a single frame and N is the number of frames that are simultaneously restored. (Typical values are N=3 and M=512.) The matrix inverse is computed in the methods discussed above using an iterative procedure which proved to be both computationally expensive and unstable in experiments.
There is a need for a cross-correlated multiframe approach to image sequence restoration which uses a cross-correlated multiframe Wiener estimate that is efficiently computed, and for a strategy of determining spectral estimates such that the cross-correlated multiframe approach is a useful and worthwhile alternative to a single-frame approach.
In addition to a cross-correlated approach, there is also a need for a motion-compensated multiframe Wiener restoration that makes use of interframe motion information that may be either available a priori, or estimated using a robust motion estimation algorithm.